<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
	<META HTTP-EQUIV="CONTENT-TYPE" CONTENT="text/html; charset=iso-8859-1">
	<TITLE></TITLE>
	<META NAME="GENERATOR" CONTENT="StarOffice/5.2 (Solaris Sparc)">
	<META NAME="AUTHOR" CONTENT=" ">
	<META NAME="CREATED" CONTENT="20011029;15444000">
	<META NAME="CHANGEDBY" CONTENT=" ">
	<META NAME="CHANGED" CONTENT="20011029;17332000">
</HEAD>
<BODY>
<table border="0" cellpadding="2" cellspacing="0" width="100%">
<tr>
<td><H2><font color="#336699" class="PageHeader">Draft - Draft - Grid Engine Enterprise Edition - policies and use cases - Draft - Draft</font></H2></td>
</tr>
</table>
<H1>1. Strict priority scheduling</H1>
<H2>a) Job based strict priority dispatching</H2>
<P>The objective of this setup is to implement a strict hierarchy of
job classes. Strict means that higher privileged jobs are started
before less privileged jobs if appropriate resources are available.
Jobs are submitted with -p &lt;priority&gt;. Scheduler uses jobs 
priority to derive per job functional share. The higher the jobs 
priority, the higher the functional share, the higher the number of 
resulting tickets.</P>
</P>
<P>HOWTO:</P>
<UL>
	<LI><P>use SHARE_FUNCTIONAL_SHARES=0 in schedd_params section of global configuration sge_conf(5)</P>
	<LI><P>specify functional tickets (e.g. 1000000) in scheduler configuration sched_conf(5)</P>
</UL>
<P>Remarks:</P>
<UL>
<P>This setup can be used as substitution for the -p &lt;priority&gt;
switch in SGE. If no other scheduling scheme is needed the use of SGE is 
recommended instead.</P>
</UL>
<H2>b) Project based strict priority dispatching</H2>
<P>The objective of this setup is to implement a strict hierarchy of
job classes. Strict means that jobs of higher privileged job classes are started
before less priviledged jobs if appropriate resources are available. Jobs are submitted 
into a job class by submitting into a corresponding project</P>
<P>HOWTO:</P>
<UL>
	<LI><P>use SHARE_FUNCTIONAL_SHARES=0 in schedd_params section of sge_conf(5)</P>
	<LI><P>specify functional tickets (e.g. 1000000) in scheduler configuration sched_conf(5)</P>
	<LI><P>add one project(5) for each scheduling relevant project</P>
	<LI><P>the hierarchy between different job classes is expressed by assigning different 
   functional shares to each project</P>
   <UL>
      <P>project_high    (90)</P>
      <P>project_medium  (60)</P>
      <P>project_low     (30)</P>
   </UL>
	<LI><P>access to higher privileged projects can be controlled via project acl/xacl</P>
</UL>
<H1>2. Functional policy - control resource shares at any time</H1>
All the setups in this category have in common that the functional policy is 
used as main policy. Setups using the functional policy as main policy ensure that 
a certain share is guaranteed to each user, project or department at any time, i.e. 
jobs of users, projects or departments that have occupied less resources than 
supposed (functional share) are prefered when dispatching jobs to idle resources. 
All the same full resource utilization is guaranteed, because unused share proportions 
are distributed among those users, projects and departments who need the resources. 
Past resource consumption is not taken into account.
<H2>a) User based functional scheduling</H2>
<P>The objective of this setup is a certain share assignment of all
the resources combined in the SGEEE cluster to different users. FCFS
scheduling is used among jobs of the same user.</P>
<P>HOWTO:</P>
<UL>
	<LI><P>use SHARE_FUNCTIONAL_SHARES=1 in schedd_params section of global configuration sge_conf(5)</P>
	<LI><P>specify functional tickets (e.g. 1000000) in scheduler configuration sched_conf(5)</P>
	<LI><P>add one user(5) for each scheduling relevant user</P>
	<LI><P>assign functional shares to each project</P>
   <UL>
      <P>userA     (10)</P>
      <P>userB     (20)</P>
      <P>userC     (20)</P>
      <P>userD     (50)</P>
   </UL>
</UL>
<H2>b) Project based functional scheduling</H2>
<P>The objective of this setup is a certain share assignment of all
the resources combined in the SGEEE cluster to different projects.
FCFS scheduling is used among jobs of the same project.</P>
<P>HOWTO:</P>
<UL>
	<LI><P>use SHARE_FUNCTIONAL_SHARES=1 in schedd_params section of global configuration sge_conf(5)</P>
	<LI><P>specify functional tickets (e.g. 1000000) in scheduler configuration sched_conf(5)</P>
	<LI><P>add one project(5) for each scheduling relevant project</P>
	<LI><P>access to higher privileged projects can be controlled via project acl/xacl</P>
	<LI><P>assign functional shares to each project</P>
   <UL>
      <P>projectA     (55)</P>
      <P>projectB     (45)</P>
   </UL>
</UL>
<H2>c) Department based functional scheduling</H2>
<P>The objective of this setup is a certain share assignment of all
the resources combined in the SGEEE cluster to different departments.
FCFS scheduling is used among jobs of the same department.</P>
<P>HOWTO:</P>
<UL>
	<LI><P>use SHARE_FUNCTIONAL_SHARES=1 in schedd_params section of global configuration sge_conf(5)</P>
	<LI><P>specify functional tickets (e.g. 1000000) in scheduler configuration sched_conf(5)</P>
	<LI><P>add departments for each scheduling relevant department</P>
	<LI><P>assign functional shares to each department, e.g.</P>
   <UL>
      <P>departmentA (90)</P>
      <P>departmentB (5)</P>
      <P>departmentC (5)</P>
   </UL>
</UL>
<H1>3. Share-tree policy - control resource shares over time</H1>
All the setups in this category have in common that the share-tree policy is 
used as main policy. Setups using the share-tree policy as main policy ensure
that a certain share is guaranteed to the instances configured in the share-tree
over time, i.e. jobs associated to share-tree branches where less resources
were consumed in the past than supposed (share-tree share) are prefered when
dispatching jobs to idle resources. All the same full resource utilization is 
guaranteed, because unused share proportions are still available for pending 
jobs associated to other share-tree branches.
<H2>a) Project based share-tree scheduling with FCFS within each project</H2>
<P>The objective of this setup is to guarantee over time a certain
share assignment of all the resources combined in the SGEEE cluster
to different projects. A FCFS scheduling is used among jobs of the
same project.</P>
<P>HOWTO:</P>
<UL>
	<LI><P>specify share tree tickets (e.g. 1000000) in scheduler configuration sched_conf(5)</P>
	<LI><P>add one project(5) for each scheduling relevant project</P>
	<LI><P>setup a share tree using qmon(1) reflecting the structure of all scheduling
	relevant projects as leafs</P>
	<LI><P>assign share tree shares to the projects</P>
<PRE>
root ------ projectA (75)
 +--------- projectB (25)
</PRE>
</UL>
<H2>b) Project based share-tree scheduling with equal share for each user within each project</H2>
<P>The objective of this setup is to guarantee over time a certain
share assignment of all the resources combined in the SGEEE cluster
to different projects. An equal share is targeted among the jobs
competing for resources within the same project.</P>
<P>HOWTO:</P>
<UL>
	<LI><P>specify share tree tickets (e.g. 1000000) in scheduler configuration sched_conf(5)</P>
	<LI><P>add one user(5) for each scheduling relevant user</P>
	<LI><P>add one project(5) for each scheduling relevant project</P>
	<LI><P>setup a share tree using qmon(1) reflecting the structure of all scheduling
	relevant projects as nodes</P>
	<LI><P>assign share tree shares to the projects</P>
<PRE>
root ------ projectA (75) ------- default (10)
 +--------- projectB (25) ------- default (10)
</PRE>
	<LI><P>add the user &quot;default&quot; as leaf in the tree below
	each of these projects</P>
</UL>
<H2>c) Project based share-tree scheduling with per user individual
shares within each project</H2>
<P>The objective of this setup is to guarantee over time a certain
share assignment of all the resources combined in the SGEEE cluster
to   different projects. Individual share assignments per user is
needed.</P>
<P>HOWTO:</P>
<UL>
	<LI><P>specify share tree tickets (e.g. 1000000) in scheduler configuration sched_conf(5)</P>
	<LI><P>add one user(5) for each scheduling relevant user</P>
	<LI><P>add one project(5) for each scheduling relevant project</P>
	<LI><P>setup a share tree using qmon(1) reflecting the structure of all scheduling
	relevant projects as nodes</P>
	<LI><P>assign share tree shares to the projects</P>
	<LI><P>add each user as leaf to the projects where they have access
	and assign individual shares</P>
<PRE>
root ------ projectA (75) ------- user1 (5)
 |                         +----- user2 (90)
 |                         +----- user3 (5)
 +--------- projectB (25) ------- user4 (30)
                           +----- user5 (30)
                           +----- user6 (30)
</PRE>
</UL>
<P>Remarks:</P>
<UL>
<P>If individual share assignment is required only for some few users the &quot;default&quot;
user can be used in combination with individual users below a project node. I.e. the tree above
can be condensed into</P>
<PRE>
root ------ projectA (75) ------- default (5)
 |                         +----- user2 (90)
 +--------- projectB (25) ------- default (30)
</PRE>
</P>
</UL>
<H1>4. Manual intervening into automated policies</H1>
<P>There are cases in which it becomes necessary for the administrator to intervene
manually, to break the automated policies (functional, share-tree) in order to prefer 
a certain group of jobs for some reason. In such cases the override policy is used by 
the administrator. One could throw in: "Why not simply changing the settings of the 
automated policy temporarily, if it's not appropriate?".</P>
<P>This is true at least for the functional policy. Not depending past 
usage, changes on the functional policy have an immediate effect on scheduling 
behavior. The share-tree policy however is significantly influenced by past usage. 
To effectuate an immediate, but temporary preferation of all jobs belonging to certain 
share-tree node entity (department, project, user) one had to make the scheduler 
"extrude past usage" temporarily. This appears to be quite circumstantial.</P>
<P>The override policy allows temporarily prefering those jobs by directly assigning 
override tickets, which is much more intuitive. Another benefit of using override 
tickets over the way of changing the automated policy is that it leaves the automated
policy setup for long-term untouched.</P> 
<P>There override policy can be used in two different variations. When the 
SHARE_OVERRIDE_TICKETS=false setting is used in the schedd_params section of 
sge_conf(5) the full ticket amount assigned to one of the object instances (user, 
department, project, job) is assigned to each corresponding single job, this is the 
default. When the SHARE_OVERRIDE_TICKETS=true setting is used these tickets are shared 
equally between all corresponding jobs.</P>
</BODY>
</HTML>
